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WO 00/27109 
VIDEO SIGMA1 PROCESSING 



This invention relates to video signal processing and in an important 
example to improved methods of processing television signals which have 
been derived from cinema film, or other camera processes having a temporal 
sampling rate lower than the field rate of the television system. The invention 
also relates more generally to the detection of motion in a video signal. 

There are several processes in which two or more fields of a television 
signal are arithmetically combined to provide a filtered or interpolated output 
signal. Examples include geometric transformation of the picture for special 
effects, aspect-ratio conversion, composite decoding and standards 
conversion. These processes can give rise to undesirable artefacts when the 
fields that are combined differ significantly because of motion, or cuts 
between different scenes. When the temporal sampling rate of the camera is 
less than the field rate of the television system the opportunity arises to 
modify the processing so that only fields corresponding to the same instant in 
time are combined. 

A particularly important example of the problem is the televising of film 
shot at 24 frames per second at a field rate of 60 fields per second. It is 
common practice to create a sequence of five television fields from two film 
frames by alternately generating two and three fields respectively from 
successive film frames. This is known as the "3:2 pull-down" technique. 
Techniques have been developed for identifying the duplicated fields by 
comparing fields one frame apart and analysing the resulting pattern to derive 
a film sequence signal. Reference is directed in this respect to US 4,881.125; 
US 4,982,280; US 4,998,287 and US 5,255,091. 

A difficulty with prior art techniques is that they rely on analysing data 
from many frames of video to identify characteristic patterns. Unless there is 
considerable delay built into the system, it is difficult to react quickly to 
changes in temporal phase. 

When film shot at 24 frames per second is televised at a field rate of 50 
fields per second, the so-called 2:2 technique is employed. Here, every film 
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frame is used to provide two video fields and the tape or film transport speed 
is modified to change the resulting 48 fields per second signal into the 
required 50 fields per second. There remains a need to identify in the video 
signal, which fields originate from the same film frame. 

5 Many processes which combine fields are arranged to operate either in 

a "video mode" where each field is assumed to correspond to a different 
moment in time, or a "film-mode" where successive fields may correspond to 
the same moment in time. It is therefore useful to create a film/video control 
signal and, in order to modify processes which combine fields inappropriately, 
10 it is helpful to derive a signal which indicates when a change in the "temporal 
phase" of the picture, or a cut to a new scene, occurs. 

A proposal has been made to detect motion between interlaced video 
fields to provide a field motion signal: see US 5,291 ,280. In this proposal, a 
signal formed by subtracting across a field delay is compared with a signal 
15 formed by subtracting across a delay of a field less one line. After filtering, 
rectifying and thresholding, the smaller of these two signals is taken as the 
field motion signal. 

Interpreting a difference taken across a field delay is complicated by 
the common practice of interlaced television scanning. The lines of 
20 successive fields are vertically misaligned by one line pitch and so, where 

vertical detail exists, the magnitude of the difference signal will not fall to zero, 
even if the fields correspond to the same scene and temporal phase. 

It is an object of this invention to provide improved method and 
apparatus that overcome certain shortcomings of the prior art. 

25 Accordingly, the present invention consists, in one aspect, in a method 

of analysing motion between adjacent fields of an interlaced video signal, 
comprising the steps of vertically interpolating one or both of the fields to 
produce respective signals for the two fields which correspond in vertical 
position; subtracting the signals to provide a field difference signal; and 

30 removing a component in the field difference signal which arises from vertical 
detail. 
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Preferably, a component in the field difference signal which arises from 
vertical detail is removed by taking a measure of vertical detail from one or 
both of the fields and subtracting either all or a proportion of the detail 
measure from the field difference signal. 

Suitably, a component in the field difference signal which arises from 
vertical detail is removed by comparing a first field difference signal provided 
for a current field and the immediately preceding field with a second field 
difference provided for a current field and the immediately succeeding field. 

In another aspect, the present invention consists in a method of 
creating a field difference signal by subtracting video signals from different 
fields characterised in that one or both of the fields are vertically interpolated 
prior to subtraction by taking weighted sums of lines from within the same field 
so as to obtain signals corresponding to similar vertical positions. 

I n yet another aspect, the present invention consists in a video filter 
process in which an output is created by taking a weighted sum of 
contributions over a filter aperture which defines the lines and fields from 
which a contribution is to be taken and the weighting of each contribution; 
comprising the steps of vertically interpolating one or more input fields to 
produce respective signals for at least two input fields which correspond in 
vertical position; subtracting the signals to provide a field difference signal for 
at least one pair of adjacent input fields; and utilising the or each field 
difference signal to select a filter aperture. 

In still a further aspect, the present invention consists in a method of 
automatically changing the operation of a video process between a film mode 
in which adjacent fields are assumed to correspond to the same point in time 
and a video mode in which adjacent fields are assumed to correspond to 

* _ ♦ i. ^ ^^r^r-tr-ioins-i trio CTPnS nf vertically interpolating one or 

dirrereni poirus in time, uuiiiuiwmy — - - 

more fields to produce respective signals for at least three input fields which 
correspond in vertical position; subtracting the signals to provide a preceding 
field difference signal for the pair of fields comprising a current field and a 
preceding field and a succeeding field difference signal for the pair of fields 
comprising the current field and a succeeding field least one pair of adjacent 
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input fields; comparing the preceding field difference signal with the 
succeeding field difference signal and changing the selection to film mode if 
the field difference signals are significantly different, and changing the 
selection to video mode if both signals are similar but not small. 

The invention will now be described by way of example with reference 
to the accompanying drawings in which: 

Figure 1 is a block diagram of a system according to one embodiment 
of the present invention for generating a film difference signal from a 
625-line interlaced television signal. 

Figure 2 is a diagram similar to Figure 1 illustrating a modification. 

Figure 3a is a diagram showing how a signal from the previous field 
can be interpolated to give a signal corresponding to the vertical 
position of the current field. 

Figure 3b is a diagram showing how signals from both the current and 
the previous fields can be interpolated to bring them into alignment with 
each other. 

Figure 4 is a block diagram of a system according to one embodiment 
of the present invention for generating an interpolation control signal. 

Figure 5 shows block diagram of a system according to one 
embodiment of the present invention for identifying film or video 
material. 

Figure 6 shows the generation of an alternative global detail signal. 

Figure 7 is a block diagram of a system according to a further 
embodiment of the present invention. 

Referring to Fig. 1: An interlaced, 625-line input video signal (1) is 
delayed by 312 lines to produce video signal (2) corresponding to the 
previous field. This signal is interpolated by averaging (3) across a one-line 
delay so as to make a signal (4) corresponding to the same vertical position 
as the (interlaced) current input line. This is subtracted from the input signal 
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and the absolute value of the result taken to give an uncorrected field 
difference signal (5) having a magnitude which increases with the difference 
in content between the current and previous fields. 

The vertical interpolation of the previous field to align it with the current 
field is shown diagrammatically in Fig. 3a. The lines of the current and 
previous fields are indicated by crosses, and their relative vertical pos.t.ons 
are indicated by their vertical positions on the diagram. The position of the 
interpolated line is shown by a circle. 

Returning to Fig. 1. a local measure of the vertical detail in the previous 
field (7) is taken by subtracting (6) across the one-line delay and taking the 
absolute value of the result. This is multiplied (8) by a constant k1, which .s 
chosen to optimise the operation of the circuit. A suitable value for k1 may be 
in the region of 0.5. 

The resulting local detail correction signal is subtracted (9) from the 
uncorrected field difference signal and clipped (10) in a threshold circuit which 
replaces values which are more negative than a predetermined threshold by 
the value of the threshold. This corrected field difference signal (11) can be 
used to find cuts or changes in temporal phase of the incoming video. 

The signal 1 1 can be further improved by integrating over all. or a 
substantial part, of the picture area (12) to make a global difference signa., 
and carrying out a similar process on the local detail signal (7) to make a 
global detail signal (13). This is multiplied (14) by a constant k2 (which 
determined in a similar way as k1 ) and subtracted from the global difference 
signal to generate an improved field difference signal 1 5. 

Another way of generating the uncorrected field difference signal and 
the local detail signal is shown in Fig. 2. 

The input 625-line interlaced video signal (1) is interpolated (21) to 
produce a signal which is vertically shifted by half of one (picture) line pitch. A 
second interpolator (22) applies an equal shift in the opposite direction to the 
signal from the previous field. The two interpolated signals are subtracted (23) 
and the absolute value of the result taken to obtain an improved local 
difference signal 27. 
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The interpolation of the two signals to bring them into vertical alignment 
is shown diagrammatically in Fig. 3b. 

Because the interpolation processes need signals from more than one 
line from the current and the preceding field (two from each in the example 
5 shown in Fig. 2), it is possible to generate local detail signals from each of 
these fields (24 and 25) by taking the absolute values of vertical difference 
signals. The two local detail signals are averaged (26) and the result used to 
correct the improved local difference signal 27. 

The local detail signal 28 and the local difference signal 27 can replace 
10 the signals 7 and 5 respectively in Fig. 1 and be processed as shown on the 
right hand side of the figure to create an improved field difference signal. 

Although the detail correction which has been described above 
considerably reduces the adverse effect of detail on the field difference signal, 
it does not eliminate it completely and further processing will now be 
1 5 described making use of the film sequence signals from more than one field at 
a time. 

In many applications there is a need to use a field difference signal to 
control whether the current field is combined with the previous field or with the 
next field. The generation of a suitable control signal is shown in Figure 4. A 

20 video signal (41 ) is input to a film sequence signal generation process (42), 
which may, for example, be that shown in Figure 1 , and a film sequence 
signal (43) obtained which gives a measure of the temporal difference 
between two fields. This signal is delayed by substantially one field in a 
suitable delay device (44) to obtain a measure of the temporal difference 

25 between the two previous fields (45). A magnitude comparison is made (46) 
between the input and the output of the delay device and the resulting signal 
(47) is used to control an interpolation process as follows: 

Let us describe the field which is being input to the block (42) as the 
next field. The film sequence signal (43) corresponds to the temporal 

30 difference between this field and the current field and the delayed sequence 
signal (45) corresponds to the temporal difference between the current field 
and the previous field. When the signal (47) indicates that the magnitude of 
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the undelayed film sequence signal (43) is significantly greater that of the 
delayed film sequence signal (45) the interpolation process combines the 
current field with the previous field; otherwise the current field is combined 
with the next field. 

Most applications which make use of film sequence information also 
have a video mode and it is helpful for the mode selection to be automatic. 
This may be achieved by processing a film sequence signal as shown in 
Figure 5. 

A film sequence signal indicating the magnitude of the temporal 
difference between successive fields dl is delayed by substantially one field in 
the delay device (501) to produce a delayed film sequence signal d 2 . The 
signals * and d 2 are fed to amplifiers (502) and (503) respectively to obtain 
amplified film sequence signals (504) and (505). The gains of the amphfiers 
are substantially equal at a value chosen to optimise the operation of the 
system; a gain of approximately two times has been found to be satisfactory. 

The delayed and amplified signal (505) has a first DC threshold value 
subtracted from it in the subtracter (506) and d, is compared with the result ,n 
the comparator (507) to produce a logic signal (508) which is active when dl 
is the greater. The comparator (509) performs an equivalent process, but wrth 
the roles of di and d 2 reversed, to produce a logic signal (510). These two 
logic signals are combined in an OR-gate (51 1 ) and the result fed to the SET 
input of a latch (512). 

The effect of this processing is to set the latch whenever there are 
significant differences between the delayed and undelayed film sequence 
signals. 

The delayed and amplified film sequence signal (505) has a second 
DC threshold subtracted from it in the subtracter (513) and the result is 
compared with dl in the comparator (514) to produce a logic signal (515) 
which is active when d, is the smaller. The comparator (516) performs an 
equivalent process, but with the roles of 4 and d 2 reversed, to produce a 
logic signal (517). These two logic signals are combined in an AND-gate (518) 
and the result fed to the RESET input of the latch (51 2). 



WO 00/27109 



-8- 



PCT/GB99/03641 



The effect of this processing is to reset the latch whenever the delayed 
and undelayed film sequence signals are of similar magnitude and greater 
than the magnitude of the second DC threshold. 

The output (519) from the latch is used to put a video process into film 
5 mode when it is active, and to change to video mode when it is not. 

Experience has shown that for some, very sharp pictures the methods 
of detail correction described above are insufficient to prevent stationary film 
images from being falsely detected as video. This difficulty can be overcome 
by generating an improved detail signal as shown in Figure 6. 

1 0 An input video signal (61 ) is delayed a total of two fields by the delay 

elements (62), (63) and (64) and the undelayed and the two-field-delayed 
signals are averaged (65). An interpolator (66) uses two or more contributions 
from the line delays (63) to create an interpolated field signal (67) which is 
aligned with the average signal. The subtractor (68) subtracts the interpolated 

15 signal from the average signal to create a detail signal which is integrated (69) 
over all, or a substantial part, of the picture area to create an improved global 
detail signal (70). 

This improved global detail signal can replace the signal (13) In 
Figure 1 , or a combination of the improved signal and the signal (1 3) can be 
20 used to correct the global difference signal. 

It has been recognised by the inventor that a component in a field 
difference signal which arises from vertical detail can be removed, not only by 
subtracting an explicit detail signal but also by comparing field difference 
signals from neighbouring pairs of fields which can be assumed to have 
25 similar amounts of vertical detail. 

Thus, turning to Figure 7, there is shown an arrangement in which an 
input interlaced 625 line video signal is passed through a 312 line delay (80), 
a 1 line delay (82) and a further 312 line delay (84). 

Signals taken across the line delay (82) are added at (86) and halved 
30 at (88) to provided an interpolated signal which correspond in vertical position 
with lines from the succeeding and preceding fields. Preceding and 
succeeding field differences are taken in subtracters (90) and (92) and the 
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absolute values taken in blocks (94) and (96). The resulting signals are 
made available to processor (98) which as described above can determine 
which of the preceding or succeeding fields has the lowest field difference. 
This determination can be made pixel by pixel or signals can be appropriately 
aggregated over a region of a field or an entire field. 

If it is desired also to distinguish between "true" video and film 
originating video, a signal can be taken from subtracter (100) across the line 
delay (82) providing a local measure of vertical detail. The abs6lute value of 
this detail signal from block (102) is then subtracted at (104) and (106) from 
the respective field difference signals. A determination can then be made of 
whether a multiple (which may be 2) of one detail corrected difference signal 
exceeds the other by a predetermined threshold. 

Although the invention has been described in terms of 625-line, 
interlaced signals, it will be appreciated by those skilled in the art that it is 
applicable to other formats, including 525-line and high-definition formats. 

The field difference signals that this invention provides can be used to 
controls a wide variety of video processes, including standards conversion, up 
or down conversion, aspect ratio conversion, special effects generators, 
composite decoders and compression encoders and decoders. 
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CLAIMS 

1 . A method of analysing motion between adjacent fields of an interfaced 
video signal, comprising the steps of vertically interpolating one or 
both of the fields to produce respective signals for the two fields which 
correspond in vertical position; subtracting the signals to provide a 
field difference signal; and removing a component in the field 
difference signal which arises from vertical detail. 

2. A method according to Claim 1 in which a component in the field 
difference signal which arises from vertical detail is removed by taking 
a measure of vertical detail from one or both of the fields and 
subtracting either all or a proportion of the detail measure from the 
field difference signal. 

3. A method according to Claim 2 in which a measure of local detail is 
used to correct the field difference signal. 

4. A method according to Claim 2 or Claim 3 in which a measure of 
global detail is derived by summing the local detail from one or both 
fields over all or a substantial part of the picture and all or a proportion 
of the global detail signal is used to correct the field difference signal. 

5. A method according to Claim 1 in which a component in the field 
difference signal which arises from vertical detail is removed by 
comparing a first field difference signal provided for a current field and 
the immediately preceding field with a second field difference provided 
for a current field and the immediately succeeding field. 
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A method of creating a field difference signal by subtracting video 
signals from different fields characterised in that one or both of the 
fields are vertically interpolated prior to subtraction by taking weighted 
sums of lines from within the same field so as to obtain signals 
corresponding to similar vertical positions. 



7. A method according to Claim 6 in which the result of the subtraction is 
corrected by taking a measure of vertical detail from one or both of the 
fields and either all or a proportion of the detail measure is subtracted 
from the field difference signal. 

8. A method according to Claim 7 in which a measure of local detail is 
used to correct the field difference signal. 

9. A method according to Claim 7 or Claim 8 in which a measure of 
global detail is derived by summing the local detail from one or both 
fields over all or a substantial part of the picture and all or a proportion 
of the global detail signal is used to correct the field difference signal. 

10. A method according to any one of the preceding claims in which the 
field difference signal is summed over all or a substantial part of the 
picture to create a global difference signal and the local detail from 
one or both fields is summed over all or a substantial part of the 
picture to create a global detail signal and a corrected field difference 
signal is obtained by subtracting all or a proportion of the global detail 

• i +wa pi^Koi field difference signal. 

' 1 1 A method according to any one of the preceding claims in which a field 
difference signal is output for each pixel of the current field. 
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12. A method according to any one of the preceding claims in which a 
field difference signal is output for each of a number of regions of the 
current field. 

13. A method according to any one of the preceding claims in which a 
single field difference signal is output for the current field. 

14. Apparatus for the geometric transformation of television pictures for 
special effects in which an interpolation process is modified in 
response to a field difference signal derived in accordance with any 
one of Claims 1 to 13. 

1 5. Apparatus for aspect ratio conversion of television pictures in which an 
interpolation process is modified in response to a field difference 
signal derived in accordance with any one of Claims 1 to 13. 

16. Apparatus for standards conversion of television pictures in which an 
interpolation process is modified in response to a field difference 
signal derived in accordance with any one of Claims 1 to 13. 

17. An interpolation process in which an output is created by taking a 
weighted sum of contributions from one or more selected input fields; 
comprising the steps of vertically interpolating one or more input fields 
to produce respective signals for at least two input fields which 
correspond in vertical position; subtracting the signals to provide a 
field difference signal for at least one pair of adjacent input fields; and 
utilising the or each field difference signal to select input fields for 
interpolation. 
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1 8 A process according to Claim 1 7 in which field difference signals are 
provided for the pair of input fields comprising a current input field and 
a preceding input field and the pair of input fields comprising the 
current input field and a succeeding input field. 

! 9 A process according to Claim 1 7 or Claim 1 8 in which a component in 
the or each field difference signal which arises from vertical detail is 
removed by taking a measure of vertical detail from one or more input 
fields and subtracting either all or a proportion of the detail measure 
from the or each field difference signal. 

20 A video filter process in which an output is created by taking a 

weighted sum of contributions over a filter aperture which defines the 
lines and fields from which a contribution is to be taken and the 
weighting of each contribution; comprising the steps of vertically 
interpolating one or more input fields to produce respective signals for 
at least two input fields which correspond in vertical position; 
subtracting the signals to provide a field difference signal for at least 
one pair of adjacent input fields; and utilising the or each field 
difference signal to select a filter aperture. 



21 



22. 



A process according to Claim 20 in which field difference signals are 
provided for the pair of input fields comprising a current input field and 
a preceding input field and the pair of input fields comprising the 
current input field and a succeeding input field. 

A process according to Claim 20 or Claim 21 in which a component in 
the or each field difference signal which arises from vertical detail is 
removed by taking a measure of vertical detail from one or more input 
fields and subtracting either all or a proportion of the detail measure 
from the or each field difference signal. 
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23. A method of automatically changing the operation of a video process 
between a film mode in which adjacent fields are assumed to 
correspond to the same point in time and a video mode in which 
adjacent fields are assumed to correspond to different points in time, 
comprising the steps of vertically interpolating one or more fields to 
produce respective signals for at least three input fields which 
correspond in vertical position; subtracting the signals to provide a 
preceding field difference signal for the pair of fields comprising a 
current field and a preceding field and a succeeding field difference 
signal for the pair of fields comprising the current field and a 
succeeding field; comparing the preceding field difference signal with 
the succeeding field difference signal and changing the selection to 
film mode if the field difference signals are significantly different, and 
changing the selection to video mode if both signals are similar but not 
small. 

24. A method of video process control according to claim 23 in which the 
film difference signals are considered to be significantly different if one 
differs from a multiple of the other by more than a predetermined 
threshold. 
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